Probe description language

ABSTRACT

A high-level language can be used to abstract away programming language complexity from a user. The high-level language can be based upon and utilize some of the semantics of the programming language, but utilize keywords to call programming language functionality transparent to the user. This provides for a shorter learning curve for people who are somewhat familiar with the programming language, but might lack the knowledge or desire to personally code an entire application or method in the program language.  
     This description is not intended to be a complete description of, or limit the scope of, the invention. Other features, aspects, and objects of the invention can be obtained from a review of the specification, the figures, and the claims.

CLAIM OF PRIORITY

[0001] This application claims priority to U.S. Provisional PatentApplication No. 60/349,457, filed Jan. 18, 2002, entitled “FLEXIBLE ANDEXTENSIBLE JAVA BYTECODE,” as well as Application No. 60/349,586, filedJan. 18, 2002, entitled “PROBE DESCRIPTION LANGUAGE,” each of which ishereby incorporated herein by reference.

COPYRIGHT NOTICE

[0002] A portion of the disclosure of this patent document containsmaterial which is subject to copyright protection. The copyright ownerhas no objection to the facsimile reproduction by anyone of the patentdocument of the patent disclosure, as it appears in the Patent andTrademark Office patent file or records, but otherwise reserves allcopyright rights whatsoever.

CROSS-REFERENCED CASES

[0003] The following applications are cross-referenced and incorporatedherein by reference:

[0004] U.S. patent application Ser. No. ______ entitled “FLEXIBLE ANDEXTENSIBLE JAVA BYTECODE INSTRUMENTATION SYSTEM,” by Rajendra Inamdar,filed concurrently herewith.

FIELD OF THE INVENTION

[0005] The invention relates generally to inserting code intoapplications.

BACKGROUND

[0006] During the life cycle of a software application, there is often aneed to observe how the application is behaving as it executes undertypical running conditions. During the development phase, programbehavior can be observed using a debugger. It is not always practical touse a debugger, however, as stopping a program at a break point canchange the behavior of a program, especially in multi-threaded programs.Thus, placing traces in a program can sometimes be the only option.

[0007] Inserting debugging or tracing code in the source code of aprogram or application can have certain drawbacks. If debugging code isinserted in several places throughout an application, the source codecan become cluttered with debug logic. Also, a programmer will have toanticipate the correct places to put the debug code when inserting itinto the application. Once compiled, such debugging code cannot bechanged. This can be undesirable while dealing with publicly-releasedsoftware.

[0008] If a problem is reported against a specific release of a product,for example, the customer support staff may have to perform severaltasks to put the required diagnostic code into the application. First,it is necessary to get an exact copy of the source code from which thereleased product was built. Then, diagnostic logic must be manuallyinserted at appropriate places in the source code. The application thenneeds to be built in exactly the same manner in which the releasedproduct was built. If the added diagnostic code needs any furthertuning, it may need to be changed by hand and the whole applicationrebuilt. This process can be very tedious and time consuming.

[0009] If the problem is in a third party library or module, the sourcecode may not be available at all. In this case, altering the source codeto add diagnostic logic is simply not possible.

BRIEF SUMMARY

[0010] Although described with respect to compiled Java applications, itwill be appreciated that similar advantages of application visibility,as well as other advantages, may obtain in other applications of thepresent invention. Such advantages may become apparent to one ofordinary skill in the art in light of the present disclosure or throughpractice of the invention.

[0011] It is an object of the present invention to aid in thedevelopment, testing, and analysis of software applications.

[0012] It is further an object of the present invention to create a toolthat allows a user to analyze an application without having access tosource code.

[0013] It is further an object of the present invention to create a toolthat allows a user to analyze an application without placing anyinstrumentation code in the application.

[0014] It is further an object of the present invention to gainvisibility into the working of applications, and allow that visibilityto vary by user.

[0015] It is further an object of the present invention to be able torecreate problems in a typical execution environment, while in a testingor troubleshooting environment. It may be desirable to be able toobserve micro level behavior without having to change the originalsource code.

[0016] It is further an object of the present invention to create a toolthat allows a user to control a granularity of instrumentation.

[0017] It is further an object of the present invention to create a toolthat provides a user with a rich set of instrumentation primitives,which can be applied at various levels.

[0018] It is further an object of the present invention to create a toolthat allows a user to provide user-specific instrumentation code thatsatisfies certain constraints.

[0019] It is further an object of the present invention to create a toolthat allows a user to analyze an application without altering programsemantics.

[0020] It is further an object of the present invention to create a toolthat allows a user to analyze an application while allowing the codeinstrumentation to retain symbolic information.

[0021] It is further an object of the present invention to create a toolthat is light-weight in terms of usage as well as execution overheads.

[0022] A system and method are presented for abstracting programminglanguage complexity away from a user. Keywords are developed andutilized that are associated with functionality in the programminglanguage. Keywords can be distinguished by appending a designatedsymbol, such as a “%” symbol. A high-level language is based on thesekeywords, as well as semantics from the programming language. A user canthen utilize this high-level language in writing code. The code writtenin the high-level language can then be interpreted for use with anapplication that is capable of accepting commands in the programminglanguage. The programming language can be any appropriate programminglanguage, such as Java. A user interface can be provided through which auser can utilize and program in this high-level language. The high-levellanguage can support at least programming language primitive types anduser-specific types. This high-level language can be used to create aprobe that can be injected into a compiled application, and can passarguments to invoked methods or methods of the compiled applications.

[0023] Other features, aspects, and objects of the invention can beobtained from a review of the specification, the figures, and theclaims.

BRIEF DESCRIPTION OF THE DRAWINGS

[0024]FIG. 1 is diagram of a system in accordance with one embodiment ofthe present invention.

[0025]FIG. 2 is a flowchart showing steps of a method that can be usedwith the embodiment of FIG. 1.

DETAILED DESCRIPTION

[0026] Embodiments of tools that can be used in accordance with oneembodiment of the present invention can provide a flexible andextensible mechanism for injecting instrumentation code into compiledJava applications. A user can also implement necessary or “arbitrary”functionality at “arbitrary” injection points in the applications. Theseembodiments can work directly on compiled Java classes and libraries,such that access to the original source code is not needed.

[0027] Tools in accordance with the present invention can utilize“probes”, or units of programmable instrumentation code that can beinjected at requested locations. The locations at which probes are to beinjected are referred to as “injection points.” A collection of probescan be implemented through a probe library. A “directive file” can beused by an end user to direct the application instrumentation process.

[0028] With the help of a directive file, a user can control thelocation(s) where the probes are injected. The user can control thescope of probe injection to packages, classes, methods, and linescorresponding to original source by specifying directives in thedirective file.

[0029] Tools in accordance with embodiments of the present invention canbe used in scenarios such as debugging, application tracking, andperformance monitoring. One benefit of such a tool is that the tool caninclude a simple user interface, as the instrumentation procedure can becontrolled with a simple configuration file. Such a tool can also belightweight, as only classes specified by the user are implemented. Atool can also be extensible, as a user can create new probes as they areneeded. Probe specifications can be written at a high level, such thatthe person creating a probe does not have to deal with low-level bytecode details. A tool can also allow the invoking of arbitrary Javaruntime classes and methods in the probe body.

[0030] A tool in accordance with the one embodiment of the presentinvention acts as a tool to inject “arbitrary” code into an application,such as into compiled Java classes, under user direction. By insertinginstrumentation code after compilation, certain benefits can beobtained. One benefit arises in the fact that program source code can bekept free of debugging code clutter. Also, the locations at whichdebugging code is inserted into the application can be controlled by theuser. This allows the placement of instrumentation to be easily changed.The instrumentation code to be inserted at specified instrumentationpoints can also be controlled by the user. The code can be specifiedusing a high level, special purpose language. This language can hide anylow level details about the byte code from the user.

[0031] Such a tool can provide a generic and flexible way to injectarbitrary instrumentation probe code into compiled Java classes. Thetool allows the user to control the granularity at which instrumentationprobes are to be inserted. The scope can be limited to a group or groupsof classes. Also, within the scope of eligible classes, a user can limitthe methods into which probes are to be inserted.

[0032] Apart from controlling where probes are to be inserted, a usercan specify that which defines a probe body. A tool in accordance withone embodiment of the present invention can utilize a simple high levellanguage, such as a Probe Description Language (PDL). Probespecifications can be created in PDL, such that a probe developer can beshielded from low level details about byte code. PDL can, for instance,allow Java objects to created and used from the probe body with a syntaxsimilar to Java.

[0033] PDL can be used to develop and implement a set of probelibraries. Since the probe body can create Java objects and invokemethods on these objects, as well as on other runtime classes, it ispossible to create a wide variety of probes for specific needs. Once aprobe library is developed, the library can be reused or used by others.

[0034] In one embodiment, an end user will typically usealready-existing probe libraries, as commonly needed probes can besupplied with the tool. Simply knowing how to use the tool shouldsuffice for such users. If a custom probe needs to be created, the userperson creating the probe may need to know PDL, or at least how to use atool or interface that is capable of creating a probe using PDL.

[0035] In one embodiment, a tool is driven by a global configurationfile and a directive file. The global configuration file can be used tostore global parameter values. Global parameter values can be replacedor supplemented by variables in the directive file. The globalconfiguration file may be seldom changed by the end user, if changed atall. A directive file can be created by an end user for a specificinstrumentation session. Together, these files can identify theavailable probe libraries and the classes to be instrumented. The filescan also identify the probes that are to be applied to selected methodswithin eligible classes.

[0036] The insertion of selected probes into selected methods can beperformed by an instrumentation engine 108, as shown in FIG. 1. In thisfigure, the source code 100 is shown for illustration purposes and isnot actually needed by the tool 106. Once an application is compiled,the compiled classes 102 can be used as inputs to the tool 106.

[0037] The dotted lines in the Figure are meant to demonstrate referencelines, while the solid arrows show information flow. The compiledclasses 102 and instrumentation directives 104 are used as inputs to theinstrumentation engine 108. The instrumentation directive 104 refers tothe probes created within the probe library. The probe library 110 iswritten in PDL. The instrumentation engine 108 can interpret PDL, soonce the probes are specified in the instrumentation directive(s) theinstrumentation engine 108 can take a specified probe, find out what theprobe's instrumentation is in PDL, and inject it at the specifiedplace(s).

[0038] The user interface to this tool is the instrumentation directive104. An instrumentation directive can be implemented as a simple textfile written in XML. The file can tell the tool 106 which classes orsets of classes are to be implemented. The file can specify thegranularity at which the instrumentation is applied, such as for exampleat certain line numbers within a method. The file can also specifyseveral directives that inform the tool of which calls are to beapplied, such as calls that apply pieces of code into an application asprobes. The directives can specify which probes are to be applied atwhich locations. The tool can determine from the directive file whichprobes to apply and where to apply them.

[0039] The tool 106 itself contains the instrumentation engine 108, aprobe library descriptor 110, and the PDL 112. The PDL block is aphantom component, as it is part of the tool and not actually a separateentity. The instrumentation engine 108 is a general purpose injectionengine that can interpret information given in an instrumentationdirective 104. The probes can be contained in a probe descriptionlibrary 110, which can be deployed as part of the tool 106. Thelibraries themselves can be created in PDL.

[0040] The code to be injected into the instrumented classes of theapplication can be extracted out of the instrumentation engine 108itself. The engine can provide generic mechanisms to define that whichthe user wants to inject. The engine can allow the user to create probeswith arbitrary implementations. The instrumentation engine 108 can alsoinject probes into the instrumented classes 114 from a probe runtimelibrary 116.

[0041] Even though a user does not have access to source code, the usercan identify several components of an application to be used with thetool. For example, a user can know which classes make up an application.Since everything is contained in class files, a user can easilydetermine the class names. A user can know at least this amount ofinformation about an application. A user can then, for example,instrument all these classes across the board.

[0042] Another possibility is that a user can determine methods in thoseclasses before or after the classes are instrumented. For example, if auser injects probes that will do program tracing, the user can learnabout the methods of the program while the program is executing. Thetool can, for example, print out all methods being instrumented in aclass. A user can then selectively instrument the methods, and so on.

[0043] When instrumenting a method a user can accomplish various tasks,such as instrumenting an application so that when it executes it willprint method entries, exits, and arguments. A user can create probes oruse probes from a library that will accomplish those tasks. When a probeis inserted, there will be some code that is injected, for example, atmethod entry and method exit.

[0044] A user can also make calls to other external run-time libraries,such as to get performance data. A probe can identify time stamps andindividual milestones in a method or application, can compute thedifferences between the timestamps, and can print out elapse times. Thetool allows users to make calls to external run-time libraries which canimplement arbitrary functionality.

[0045] A method is shown in FIG. 2 that can be used with the system ofFIG. 1. The user can specify the injection points in the application atwhich the code is to be injected 200. An instrumentation engine cancompile the probe specifications in probe libraries into an internalrepresentation 202. The instrumentation engine can then interpret theprobe code into byte code 204. The instrumentation engine can theninject the appropriate byte code into method bodies at appropriateplaces 206. An instrumentation engine can provide a framework forinjecting instrumentation probes into compiled Java classes. Aninstrumentation engine can create instrumented class files from inputclass files, and can read and compile probe specifications from probelibraries. An instrumentation engine can identify and read byte code forclasses specified by user directives, and can identify injection pointswithin classes specified by the user. An injection engine can identifyprobes to be injected into specified classes and can inject probeinstrumentation code into classes at appropriate injection points. Theinjected code can be generated by compiling probes within the context ofan injection point.

[0046] Global configuration information can be stored in a globalconfiguration file, which can have the a format similar to that of thefollowing examples. Global Configuration File Example #1 <?xmlversion=“1.0” encoding=“UTF-8”?> <DOCROOT> <GLOBAL_OPTIONS> <OPTIONS><OPTION name=“optname” value=“optvalue”/>... </OPTIONS> <PROBE_LIBS><PLIB prefix=“prefix-name”value “probe-library-path”/>... </PROBE_LIBS><EXCLUDE_PATTERNS> <PATTERN value=“class-name-pattern”/>...</EXCLUDE_PATTERNS> <SAFE_VALUES> <PATTERNvalue=“class-name-pattern”/>... </SAFE_VALUES> </GLOBAL_OPTIONS></DOCROOT> Global Configuration File Example #2 <?xml version“1.0”encoding“UTF-8”?> <DOCROOT>  <GLOBAL_OPTIONS> <OPTIONS> <OPTIONname=“TRACE” value=“DEBUG”/> <OPTION name=“OUTDIR” value=“output”/><OPTION name=“itool.runtime.path” value =“c:/weblogic/lib/log4j.jar”/></OPTIONS> <PROBE_LIBS> <PLIB prefix“log4j” value=“log4j.pdl”/> <PLIBprefix“jsr47” value=“jsr47.pdl”/> </PROBE_LIBS> <EXCLUDE_PATTERNS><PATTERN value=“java.*”/> <PATTERN value=“javax.*38 /></EXCLUDE_PATTERNS> <SAFE_VALUES> <PATTERN value=“java.lang.*”/><PATTERN value=“java.util.*”/> </SAFE_VALUES>  </GLOBAL_OPTIONS></DOCROOT>

[0047] In a global configuration file similar to that shown above, anOPTIONS element can be empty or can include a number of OPTION elements.Each OPTION element can specify a global parameter. The “name” attributeof the OPTION element can identify the name of the parameter and the“value” attribute can identify its value. A global parameter specifiedin the global configuration file can be over ridden by a correspondingOPTION element in a directive file. It can be further over ridden by acommand line option.

[0048] Certain global parameters can be used by the tool. One suchparameter is TRACE. A TRACE parameter can be used to specify the tracinglevel to be used. Possible values include DEBUG, INFO, WARN, and ERROR.The WARN value can be set as the default. Another such parameter isOUTDIR, which can be used to specify the output root directory whereinstrumented classes can be created.

[0049] Another possible global parameter is itool.class.path. Thisparameter can specify the class path used to locate classes to beinstrumented. If not specified, the parameter can default to a CLASSPATHenvironmental variable.

[0050] A itool.runtime.path variable can specify the class path used tolocate runtime classes which will be used from the instrumentation code.If not specified, the variable can default to the CLASSPATHenvironmental variable. Parameter values for itool.class.path anditool.runtime.path can be specified in the same format that of a typicalCLASSPATH environmental variable.

[0051] Another element that can be included in a global configurationfile is PROBE_LIBS. A PROBE_LIBS element can be empty, or can include anumber of PLIB elements. A prefix attribute can be used to specify thename-space for the probe library. When using a prefix attribute, thenames of all probes in the probe library can be prefixed with aspecified prefix. The value attribute can specify the path to the probelibrary file. The probe library file can contain specifications to agroup of instrumentation probes. A list of available probe libraries canbe supplemented with PLIB elements in the directive file.

[0052] An EXCLUDE_PATTERNS element in the global configuration file canbe empty or can include a number of PATTERN elements. Each PATTERNelement can specify a pattern used to exclude classes from theinstrumentation process. A list of exclusion patterns can besupplemented with EXCLUDE_PATTERNS in the directive file.

[0053] A PATTERN element can specify the pattern to be used for matchingstrings. It can have many attributes, such as a TYPE attribute. Whenpresent, TYPE can have a value such as for example “re”, allowing thepattern specified by the “value” attribute to be taken as a Perl regularexpression. Otherwise, it can allow for limited wild card specification.The VALUE attribute can specify the pattern to be used for stringmatching. If TYPE's attribute value is not “re”, it can specify an exactmatch or start string if the pattern ends in “*”. For example: <!--Matches all classes starting with “com.foo.”--> <PATTERN type=“re”value=”{circumflex over ( )}com. foo.\”/> <!-- specified as Perl regularexpr. --> <PATTERN value=“com.foo.*”/> <!-- equivalent to above. --><!-- Matches com.foo.MyClass only --> <PATTERN type=“re”value=“{circumflex over ( )}com\.foo\.Myclass$”/> <!-- specified as Perlregular expr. --> <PATTERN value“com.foo.Myclass”/> <!-- Matchescom.foo.MyClass only -->

[0054] Another possible element in a global configuration file isSAFE_VALUES. There can be any number of PATTERN elements under aSAFE_VALUES element, or the element can be empty. Each pattern canspecify the class names whose “toString” method can be safely usedwithin inserted instrumentation code. The list of “safe” classes can besupplemented with SAFE_VALUES element in the directive file.

[0055] PDL can allow probe code to invoke methods on objects such asJava objects. If those methods are instrumented themselves, they canlead to undesirable consequences in certain circumstances, such asinfinite recursion or infinite loops. The SAFE_VALUES is a hint that itis OK to invoke methods on these classes and objects of these classes.This issue will be covered in more detail later in this document.

[0056] Directive File

[0057] A directive file can be used to specify which probes are to beinserted into which class files. A directive file can limit the scope ofinstrumentation to a group of classes. Additionally, a directive filecan provide a section such as GLOBAL_OPTIONS to replace or supplementinformation specified by a global configuration file.

[0058] A directive file can define an itool.runtime.path to identifyruntime classes. The directive file can also define an itool.class.pathto identify classes to be instrumented, and can define patterns toinclude or exclude classes from the instrumentation process. A directivefile can register probe libraries and specify directives to inject namedprobes into named classes. A sample directive file is shown below.Directive File Example #1 <?xml version“1.0”encoding“UTF-8”? > <DOCROOT>global options replace/supplement values specified by globalconfiguration file <GLOBAL_OPTIONS> <OPTIONS> <OPTIONname“optname”value“optvalue”/> </OPTIONS> <PROBE_LIBS> <PLIBprefix“prefix-name”value=“probe-library- path”/> </PROBE_LIBS><EXCLUDE_PATTERNS> <PATTERN value=“class-name-pattern”/></EXCLUDE_PATTERNS> <SAFE_VALUES> <PATTERN value“class-name-pattern”/><SAFE VALUES> </GLOBAL_OPTIONS> <!-- An instrument spec identifies theclasses to which a set of probes need to be applied. <INSTRUMENT_SPEC><INCLUDE_PATTERNS><!-- apply probes in this spec only to classes below--> <PATTERN value“class-name-pattern”/>... </INCLUDE_PATTERNS><EXCLUDE_PATTERNS><!-- within this spec, exclude these classes frominstrumentation --> <PATTERN value=“class-name-pattern”/>...</EXCLUDE_PATTERNS> <APPLY_PROBES><!-- apply following probes toeligible classes in this spec --> <PROBE name“probe-name”method_type=“method-type” pattern_type=“pattern-type”include_methods=“method-patterns” exclude_methods=“method-patterns”probe_arg=“probeargval”... /PROBE>... </APPY_PROBES>  </INSTRUMENT_SPEC></DOCROOT> Directive File Example #2 <?xml versiow“1.0”encoding“UTF-8”?> <DOCROOT>  <GLOBAL_OPTIONS> <OPTIONS> <OPTIONname=“TRACE” value“debug”/> <OPTION name=“itool.class.path”value=“C:/weblogic/classes”/> </OPTIONS> <PROBE_LIBS> <PLIBprefix=“log4j” value“log4j.pdl”/> </PROBE_LIBS> <EXCLUDE_PATTERNS><PATTERN value=“org.apache.*”/> </EXCLUDE_PATTERNS>  </GLOBAL_OPTIONS> <INSTRUMENT_SPEC> <INCLUDE_PATTERNS> <PATTERNvalue=“com.beasys.engtools.Webcli.*”/> <PATTERNvalue=“com.beasys.engtools.util.*”/> </INCLUDE_PATTERNS><EXCLUDE_PATTERNS> <PATTERN value“com.beasys.engtools.webcli.properties.*”/> </EXCLUDE_PATTERNS><APPLY_PROBES> <PROBE name=“methodEntry” include_methods=“*”/> <!-- notehow the String value of label is passed --> <PROBE name=“printExpr”lineno=“100-110” label=‘“Object-Name:”’expr=“objName”/> </APPY_PROBES> </INSTRUMENT_SPEC> </DOCROOT>

[0059] An INSTRUMENT_SPEC element in the directive file can direct howthe instrumentation should be performed. Each INSTRUMENT_SPEC elementcan specify the classes to which instrumentation is to be applied. Theelement INSTRUMENT_SPEC can contain one or more PROBE elements, whichspecify the probe(s) to be applied to eligible class(es). The specifiedprobes can be applied to classes listed in an INCLUDE_PATTERNS element,which are not in any EXCLUDE_PATTERNS in the global options or inEXCLUDE_PATTERNS of the current specification. A directive file can havemultiple INSTRUMENT_SPEC elements.

[0060] A PROBE element can specify the probe to be applied to eligibleclasses. It can have attributes such as NAME, which can be a requiredattribute to specify the name of the probe, which exists in one of thespecified probe libraries.

[0061] If a pattern_type attribute exists, it can have a value such as“re”. If the value is “re”, patterns used in include_methods andexclude_methods are Perl regular expressions. This is similar to what isdescribed with respect to the TYPE attribute in PATTERN elements. Thisattribute is optional in some embodiments.

[0062] A method_type attribute can be used to identify the type ofmethods within eligible classes to which a probe should be applied. Thisattribute, which can be optional, can include a list of values such as,for example:

[0063] constructor: can be applied to class constructors

[0064] private: can be applied to private methods

[0065] protected: can be applied to protected methods

[0066] public: can be applied to public methods

[0067] all: can be applied to all methods

[0068] An include_methods attribute can specify comma-separatedpatterns, in order to identify methods to which the probe is to beapplied. If this attribute is not included, all methods can be eligible.An exclude_methods attribute can also be included to identify methodswhich should be excluded. If this attribute is not included, no methodswill be excluded.

[0069] A “lineno” attribute can be specified, for example, when theprobe scope is %online. The value of lineno can specify the line numberin the original source where the probe is to be applied. The valuespecified can be a number, such as 100, or a range, such as “100-110.” Avalue of “*” can be used to imply that the probe is to be applied at alllines in the eligible method.

[0070] When the probe scope is %oncall, a “callsite” attribute can bespecified. The value of callsite can specify the pattern to identifycall sites at which the probe will be applied. For example, settingcallsite=“read” can apply the probe at the call to method “read” withinthe scope of the method being instrumented.

[0071] A probe can have zero or more arguments. When a probe hasarguments, the arguments can be specified with a statement such asProbeArg=“ProbeArgVal”. The name of the attribute should match with thename of the probe argument.

[0072] Within an eligible class, a probe can be applied to those methodswhich satisfy restrictions placed by attributes such as method_type,include_method and exclude_method. In one embodiment, if the method namematches patterns specified in exclude_methods, the probe is not applied.If the method name does not match patterns specified in include_methods,the probe is not applied. If the method type is not present in themethod_type attribute, the probe is not applied.

[0073] More than one probe can be applied to the same method in a class,due to, for example, multiple PROBE elements within the sameINSTRUMENT_SPEC element or due to multiple INSTRUMENT_SPEC elementswithin the directive file.

[0074] Principle of Operation

[0075] A high level description of the operation of a tool in accordancewith one embodiment of the present invention is as follows. A directivefile is parsed to obtain certain information, such as a list of classarchive libraries and directories containing compiled classes to beinstrumented. A list of classes specifically excluded from theinstrumentation process is also parsed, as well as libraries of probeswhich are available for instrumentation. A list of instrumentationspecifications is obtained, comprising a subset of classes which shouldbe considered for instrumentation within the scope of theinstrumentation specification. The list of specifications can alsoinclude classes to be specifically excluded from instrumentation withinthe scope of the instrumentation specification, and a list of probedirectives, used to apply probes from available probe libraries whichare to be injected in eligible classes. The probe directives can furtherrestrict the classes, methods within the classes, and locations withinthe methods where the probe can be applied. There can be multipleinstrumentation specifications, and each instrumentation specificationcan define different class subsets which are eligible forinstrumentation within the scope of the instrumentation specification.

[0076] For each class archive library and directory containing compiledclasses, valid class files can be identified as well as their fullyqualified class names. For each probe library available forinstrumentation, the probe library file can be parsed, and probesdefined in the probe library can be identified. Parsed trees can bestored for defined probes in the probe repository.

[0077] For each identified class, the tool will determine if that classis explicitly excluded or is an interface, and will skip all suchclasses. For each executable method in the class, a list of eligibleprobe directives can be identified which are to be applied to thismethod in the class, obtained from the list of probe directivesidentified for all instrumentation specifications. For each eligibleprobe directive identified, a copy of the parsed representation of thespecified probe can be obtained from the probe repository.

[0078] It can then be determined whether the formal probe parameters arespecified in the probe directive. An “oninit” section of a probe bodycan specify the code to be emitted in the static class initializer afterinstrumentation. One purpose for doing so would be to capture someobjects that might be used or re-used by different probes. If it exists,the “oninit” section can be complied to emit code in the static classinitializer. An “oninit” section can create a reusable static object,which can be available once the class loads. The static initializer canexecute when the class is being loaded. Thereafter, everything that isbeing created and initialized can be accessed from the bodies of thedifferent probes.

[0079] An “onentry” section of a probe body can specify code to beemitted at method entry. If it exists, the probe formal parameters canbe replaced with actual parameters specified in the probe directive. The“onentry” section can also be compiled to emit code at the method entry.The section can create objects and variables, and can initialize them.

[0080] An “onexit” section in the probe body can specify code to beemitted before a return from a method. If the section exists, the probeformal parameters can be replaced with actual parameters that arespecified in the probe directive. All return instructions within themethod can also be identified. For each return instruction within themethod, the “onexit” section can be compiled within the context of thereturn instruction. A generated instruction sequence can be injectedbefore the return instruction.

[0081] A “before_line” section in the probe body can specify code to beemitted before an instruction corresponding to a line in the originalsource code. If the section exists, the probe formal parameters can bereplaced with actual parameters specified in the probe directive. Allinjection points can be identified that match the “lineno” attribute inthe probe directive. For each injection point within the method, a“before_line” section can be compiled within the context of theinjection point. The generated sequence can then be injected atapplicable injection points.

[0082] A “before_call” section in the probe body can specify code to beemitted before a call to a named method. If the section exists, theprobe formal parameters can be replaced with actual parameters specifiedin the probe directive. All injection points can be identified withinthe method where the named method is invoked. For each such injectionpoint, a “before_call” section can be compiled within the context of theinjection point. The generated sequence can then be injected at theinjection point.

[0083] An “after_call” section in the probe body can specify code to beemitted after a call to a named method. If the section exists, theformal probe parameters can be replaced with actual parameters specifiedin the probe directive. All injection points can be identified withinthe method where the named method is invoked. For each such injectionpoint, an“after_call” section can be compiled within the context of theinjection point. A generated sequence can be injected at the injectionpoint.

[0084] An “oncatch” section in the probe body can specify code to beemitted at the beginning of a catch block. If the section exists, formalprobe parameters can again be replaced with actual parameters specifiedin the probe directive. All injection points involving catch blocks canbe identified. For each injection point, the “oncatch” section can becompiled within the context of the injection point. A generated sequencecan then be injected at the injection point.

[0085] If any applied probe has an “oninit” section and the class doesnot have a static initializer method, a static initializer method can becreated. Generated initialization code can be injected into the staticinitializer method. The modified class can then be saved to a designatedlocation.

[0086] Certain users should be able to create probes in PDL for specificneeds. For example, a library of probes can be created by a user toperform specific tasks, such as application tracing and logging. Aseparate library can be created for performance measurements, such asmight make use of suitable runtime libraries. Libraries of such probescan be created and shared among end users. Since PDL hides details aboutlow level bytecode and class manipulation facilities, it can be veryeasy for a user to maintain and adapt created probes. A tool inaccordance with one embodiment of the present invention can also beextended and integrated with other programs, such as integrateddevelopment environments, in order to enhance usability.

[0087] Probe Description Language

[0088] A custom language such as the Probe Description Language (PDL)can provide a high level notation to specify the body of aninstrumentation probe. It can be implemented as a special purposelanguage that allows invoking methods on Java classes and objects. PDLcan be used to write probe libraries, for example.

[0089] PDL is a language that can be, in one implementation, very closeto Java, although high-level languages abstracting away the complexityof other languages can be implemented in a similar fashion using theteachings included herein with respect to the Java language. In aJava-based embodiment, some of the semantics are the same as Java, suchas the semantics used to call methods. This lessens the learning curveneeded to understand and use PDL if a user knows basic Java commands.PDL can link classes, objects, and methods in the Java world. Forexample, in the environment for such a tool a user may need to specifywhich pieces of code will end up where, such as when a user wants toinject something at method entry and something else just before methodexit as part of the same probe.

[0090] PDL can also provide certain keywords, such as may be useful incomputing certain information. For example, a user can use keywords todetermine which class is currently running in an application. A user canalso determine the current method being instrumented or the current linenumber. Certain features can be included that are tailored towardexpressing the type of information that the user would like to evaluateas part of the probe.

[0091] PDL can provide a mechanism to describe instrumentation code. PDLcan support the use of Java primitive types, as well as user-specifictypes. PDL can support simple arithmetic operations and stringcatenation. PDL can allow the creation of static objects and variables,as well as objects and variables on stack. PDL can allow calls toexternal Java runtime library methods and the creation of parameterizedprobes.

[0092] PDL can provide constructs to allow a probe writer to declare andinstantiate variables of any Java type in the class static initializer.PDL can also allow a probe writer to declare and instantiate variablesof any Java type on the stack. PDL can also allow for arithmeticoperations. Literal values of Java primitive types can be defined andutilized, as well as literal String values. PDL can allow for stringconcatenations, and can provide access to classes, objects, andvariables within an application being instrumented. This approach isconsistent with Java scoping and access rules. PDL can be used to invokemethods on arbitrary Java runtime classes, and can allow a probe writerto pass arguments to invoked methods. Any accessible objects andvariables can be passed as arguments, whether originating from theapplication or created in the probe. PDL can allow for conditionalexecution. PDL can also allow a probe writer to create parameterizedprobes, such that the same probe can be used in different contexts tooperate on different arguments.

[0093] PDL can support comments or comment statements, such as C++ stylecomments. For example, anything following a double slash, or “//”, canbe treated as a comment. Anything between a “/*” and a “*/”, possiblyfollowing a “*”, can also be treated as a comment.

[0094] In order to specify that a word is a keyword in PDL, all keywordscan start with a specific symbol, such as starting with a “%” character.The keywords can also have specified meaning based on their context.Exceptions can include Java primitive types such as “int” and “long”.Following are keywords and possible meanings for one embodiment of thelanguage. The primary data types in this embodiment are byte, char,short, int, long, float, double, and boolean.

[0095] %pdl—probe description library

[0096] %probe—probe specification

[0097] %file—source file name corresponding to the class(java.lang.String)

[0098] %class—fully qualified class name being instrumented(java.lang.String)

[0099] %method—current method name being instrumented (java.lang.String)

[0100] %line—current line number, or line number in the original sourcecode where the code or probe will be injected (int)

[0101] %retval—return value of the method

[0102] %argc—method parameter count

[0103] %argv—array of method parameters or arguments

[0104] %oninit—code to be emitted in static initializers

[0105] %onentry—code to be emitted at method entry

[0106] %onexit—code to be emitted at method exit

[0107] %before_line—code to be emitted before specified source line

[0108] %before_call—code to be emitted before call to specified method

[0109] %after_call—code to be emitted after call to specified method

[0110] %oncatch—code to be emitted in a catch block

[0111] %return—a method return value, can be valid only with the“onexit” section

[0112] %caught—a caught exception, can be valid only within the“oncatch” section

[0113] Using keywords such as %file, %class, %method, %line, and %retvalin the probe body can yield corresponding values of typejava.lang.String. These strings can participate in string concatenationto form other relevant strings. These keywords can also be combined withother variables and literal values and passed to appropriate runtimemethods for further processing. The syntax and semantics of PDL can alsobe extended by a sufficiently skilled probe writer.

[0114] PDL need not impose restrictions on which Java classes andmethods can be invoked within the probe code. Therefore, it can bepossible to invoke arbitrary runtime classes to implement arbitraryfunctionality.

[0115] During an instrumentation process, probe sections can be compiledand generated code can be injected at designated injection points.Appropriate syntactic and semantic analysis can be performed within thecontext of each injection point to ensure that the probe body is validwithin the context of each injection point. If such checks fail, anappropriate message can be printed and the probe kept from beinginjected.

[0116] Probes can be created to accept parameters. A specific instanceof a probe directive using such a probe would pass the actual parametersas named attributes. Formal parameters can be replaced with acutalparameters while processing a probe directive at every probe injectionpoint. This feature can allow the creation of generic probes which willinject appropriate code, based on the passed parameters.

[0117] The following example illustrates how a probe can be written toprint the value of an expression at a given line. %probe printExprAtLine(label, expr) { %before_line { java.lang.System.out.println( (((“IVTprintExprAtLine ”+ %class +“::” + %method + “@Line +%line”+“”)+ label) +expr) ); } }

[0118] Multiple probes can be written in a single file to create a probelibrary. Multiple probe libraries can be created and made available tothe system for application instrumentation.

[0119] A user can inject specific probes at specific locations bycreating a directive file. A directive file can be implemented as asimple XML file. Elements within the directive file can identify the setof classes eligible for instrumentation, the set of classes to beexecuted for instrumentation, the set of probe libraries available, andthe list of probes to be injected in eligible classes at eligiblelocations.

[0120] A PDL program can specify a probe library. One example of theform of a PDL program, or probe library, is given by: %pdlprobe-library-name;//identifies the probe library name %probe probe-name([probe-args...]) { %oninit{ stmt-list } %onentry { stmt-list } %onexit{ stmt-list } %before_line { stmt-list } %before_call { stmt-list }%after_call { stmt-list } %oncatch { stmt-list } }

[0121] A probe can be uniquely identified by a library name and a probename. A probe arguments list, such as may be referred to as“probe-args”, can be implemented as a comma-separated list of probearguments. The arguments can be used within the body of the probesymbolically. For a specific probe insertion, the values of the probearguments can be specified by the corresponding named attributes in aPROBE element of the directive file. The probe body can have one or moresections.

[0122] A probe can contain any combination of sections, but sectionssuch as “onexit”, “before_line”, “before_call”, “after_call”, and“oncatch,” described above, may need to be mutually exclusive. Each ofthese sections can be empty, or they can contain a number of statements.Each statement can end with a symbol such as a semi colon.

[0123] One type of statement that can be supported is a declaration.Variables can be declared using a declaration statement of the form:

[0124] type name[=initializer-expression];

[0125] All Java types can be supported, including primitive types anduser types. An optional “initializer” can initialize a newly-createdvariable. Variables created in an %oninit section can be implicitlystatic. Variables created in an %onentry section can be created on thestack, and the scope of these variables can be entire method body. Thesevariables can not be declared in other sections.

[0126] Variables declared within probes can be internally renamed usinga name-altering scheme, such that the variables do not clash withvariables in the application being instrumented. For example:

[0127] int index;

[0128] java.util.Vector vector=new java.util.Vector( );

[0129] long t0=100;

[0130] Another statement that can be supported in PDL is an assignment.Assignment statements can be used to assign values to variables, and cantake a form such as “name=expression;” given by the example“elapsed_time=(t1−t0);” statement.

[0131] Another supported statement can be an expression. PDL expressionscan support simple arithmetic operations, such as the addition andsubtraction of integer types. Expressions can also support stringcatenation, method calls on Java classes and objects, and “new”operators to instantiate objects. An example is given by the statement“vector=new java.util.Vector( );”.

[0132] Methods on Java classes, interfaces, and objects can be invokedwith the same syntax as in Java. Passed parameter types can matchcorresponding method signatures. The tool can report an error if theparameters and signatures do not match. Values returned from such methodcalls can be assigned to variables of compatible types. The objects onwhich the methods are invoked can be probe variables, as declared in theprobe, or they can come from the application being instrumented.

EXAMPLE

[0133] t0=java.lang.System.currentTimeMillis( );

[0134] java.lang.System.out.println(“ElapsedTime:”+(java.lang.System.currentTimeMillis( )−t0));

[0135] In some embodiments, variables can be declared only in %oninitand %onentry sections. Variables declared in an %oninit section can beimplicitly static, and variables declared in an %onentry section can becreated on a stack. It is also possible to not have any sections in aprobe body. Such a probe can be treated as an empty probe. Access tovariables from an application being instrumented can be required tosatisfy normal Java scoping and access rules. Since the scope of a probecan be implicitly identified by analyzing the probe body, it may not benecessary to track the scope of a probe in the PDL syntax.

[0136] Interface

[0137] Instead of creating a directive file by hand, or creating an XMLfile directly, a user interface can be supplied which will help a userto generate a directive file. This can further lower the amount ofknowledge required to inject probes into a compiled application.

[0138] A user interface can take the form of, for example, apoint-and-click graphical user interface (GUI) or an identificationtree. If using an identification tree, a user can click on nodes withinthe tree, with nodes representing, for example, classes and methods inthe application. The user can simply select a point at which to insert aprobe. Alternatively, a user can click through lists of methods andclasses to select a position through a GUI, then select a probe from alist and specify parameters or options that might be needed by theprobe.

[0139] Architecture Overview

[0140] As discussed above, a user input to this tool can consist of aset of compiled class files along with a file containing instrumentationdirectives. Input class files can be manipulated by the tool to producefunctionally equivalent instrumented classes. The instrumentationdirectives can be written using probes from available probe libraries.The probe libraries can be created by generating probe librarydescriptor files in PDL. It is possible to create and deploy multipleprobe libraries.

[0141] The instrumentation engine can read input class files andinstrumentation directives provided by the user. The instrumentationengine can refer to the probe library descriptors to identify theinstrumentation specification for the probes. Using the instrumentationspecifications, the instrumentation engine can emit appropriatebyte-code within the class files to produce instrumented class files.

[0142] The instrumented classes can be executed with the providedruntime library to generate runtime traces. First, the command line,global configuration, and directive file can be processed. Probelibraries can then be compiled in order to build an internalrepresentation for all available probes. A semantic analysis of eachprobe specification can be completed. Eligible classes are identifiedfor instrumentation.

[0143] For each eligible class, the class file can be read and a staticinitializer method created, if needed. For each method in the class, allprobes can be identified which need to be applied to the method. Foreach probe to be applied, the probe application location is identified.Code in the static class initializer is emitted, if needed, based on thespecification in the %oninit section. Code in the method body is emittedat the indicated location based on different sections in the probe body.The modified class file can then be written.

[0144] Libraries

[0145] A user can make calls from the tool to a third-party library,such as a Log4j API, in order to accomplish tasks such as loggingmessages into the console. Using Log4j provides a flexible way ofsending messages to different destinations. A user can insert callswithin the source code and make calls to this API. A probe library canbe created around this API that can be used for application tracing. Theprobe library can be a bridge between the application the user wants toinstrument and the Log4j API. There can be a run-time library that doesvery specific things, such as application tracing and performancemonitoring. The user can then leverage this run-time library by creatingprobes that make a bridge to that library. The API can be called withouthaving to alter the source code.

[0146] Industrial Application

[0147] Embodiments in accordance with the present invention will allowdevelopment engineers to accomplish tasks such as identifyingconcurrency problems and race conditions. Development engineers can alsouse such tools to monitor the state of a program as it executes andanalyze software logic. Embodiments in accordance with the presentinvention will also allow performance engineers and the like to makeperformance measurements at the micro level and do competitive analysis.Performance engineers can also do a statistical analysis over microlevel measurements to expose bottlenecks and load imbalances.

[0148] The foregoing description of preferred embodiments of the presentinvention has been provided for the purposes of illustration anddescription. It is not intended to be exhaustive or to limit theinvention to the precise forms disclosed. Many modifications andvariations will be apparent to one of ordinary skill in the art. Theembodiments were chosen and described in order to best explain theprinciples of the invention and its practical application, therebyenabling others skilled in the art to understand the invention forvarious embodiments and with various modifications that are suited tothe particular use contemplated. It is intended that the scope of theinvention be defined by the following claims and their equivalence.

What is claimed is:
 1. A system for abstracting low level instructioncomplexity away from a user, comprising: a high-level language based onthe semantics of a programming language, said high-level languagecomprising keywords that represent functionality in the programminglanguage; and a mechanism for generating programming language code fromsaid high-level language.
 2. A system according to claim 1, wherein saidhigh-level language further comprising a user interface allowing a userto program in said high-level language.
 3. A system according to claim1, wherein said high-level language allows invoking methods on Javaclasses and objects.
 4. A system according to claim 1, wherein saidhigh-level language allows linking of classes, objects, and methods. 5.A system according to claim 1, wherein said high-level language providesa mechanism to describe instrumentation code.
 6. A system according toclaim 1, wherein said high-level language supports native programminglanguage types and user-specific types.
 7. A system according to claim1, wherein said high-level language allows a user to pass arguments toinvoked methods without having any knowledge of the workings of thosemethods.
 8. A system according to claim 1, wherein said high-levellanguage provides for the creation of parameterized probes that can beused in different contexts to operate on different arguments.
 9. Asystem according to claim 1, wherein said high-level language specifiesthat a word is a keyword by appending a symbol to the word.
 10. A methodfor abstracting programming language complexity away from a user,comprising: developing keywords that are associated with functionalityin the programming language; developing a high-level language based onthe keywords and semantics from the programming language; allowing auser to utilize the high-level language in writing code; andinterpreting the high-level language for use with an application capableof accepting commands in the programming language.
 11. A methodaccording to claim 10, wherein said programming language is Java.
 12. Amethod according to claim 10, further comprising the step of specifyingkeywords by appending a designated symbol.
 13. A method according toclaim 10, further comprising the step of providing the user with a userinterface through which the user can utilize the high-level language.14. A method according to claim 10, further comprising the step ofsupporting programming language primitive types and user-specific typesin the high-level language.
 15. A method according to claim 10, furthercomprising the step of creating a probe using the high-level languagethat can be injected into a compiled application.
 16. A method accordingto claim 10, further comprising the step of allowing the user to passarguments to invoked methods using the high-level language.
 17. A methodaccording to claim 10, further comprising the step of altering any namein the high-level language that might conflict with a name in theprogramming language.
 18. A computer-readable medium, comprising: meansfor developing keywords that are associated with functionality in theprogramming language; means for developing a high-level language basedon the keywords and semantics from the programming language; means forallowing a user to utilize the high-level language in writing code; andmeans for interpreting the high-level language for use with anapplication capable of accepting commands in the programming language.19. A computer program product for execution by a server computer forabstracting programming language complexity away from a user,comprising: computer code for developing keywords that are associatedwith functionality in the programming language; computer code fordeveloping a high-level language based on the keywords and semanticsfrom the programming language; computer code for allowing a user toutilize the high-level language in writing code; and computer code forinterpreting the high-level language for use with an application capableof accepting commands in the programming language.
 20. A system forabstracting programming language complexity away from a user,comprising: means for developing keywords that are associated withfunctionality in the programming language; means for developing ahigh-level language based on the keywords and semantics from theprogramming language; means for allowing a user to utilize thehigh-level language in writing code; and means for interpreting thehigh-level language for use with an application capable of acceptingcommands in the programming language.
 21. A computer system comprising:a processor; object code executed by said processor, said object codeconfigured to: develop keywords that are associated with functionalityin the programming language; develop a high-level language based on thekeywords and semantics from the programming language; allow a user toutilize the high-level language in writing code; and interpret thehigh-level language for use with an application capable of acceptingcommands in the programming language.
 22. A computer data signalembodied in a transmission medium, comprising: a code segment includinginstructions to develop keywords that are associated with functionalityin the programming language; a code segment including instructions todevelop a high-level language based on the keywords and semantics fromthe programming language; a code segment including instructions to allowa user to utilize the high-level language in writing code; and a codesegment including instructions to interpret the high-level language foruse with an application capable of accepting commands in the programminglanguage.